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Abstract 

In this report we discuss the treatment of statistical errors in cut ef- 
ficiencies. The two commonly used methods for the calculation of the 
errors, Poissonian and Binomial, are shown to be defective. We derive the 
form of the underlying probability density function and characterize its 
mean, mode, and variance. A method for the calculation of errors based 
on the variance of the distribution is discussed. 



1 Introduction 

In many areas of experimental particle and nuclear physics the efficiencies of the 
detectors used to record the particles is evaluated by Monte Carlo simulations. 
These simulations usually incorporate a generator that produces particles and 
events with known parameters and a detailed simulation of the detectors and 
their respective response. The produced Monte Carlo data is then run through 
the standard event reconstruction software and the result is compared to the 
input from the generator. The ratio of output over input defines the so called 
response function of the detector. This function is then used to correct the result 
obtained from the reconstruction of the real data yielding the final corrected 
spectra. The response function depends in general on many parameters, e.g., 
transverse momentum and rapidity of the particle, event multiplicity and much 
more. 

From a statistics point of view this procedure can be simplified to the comparison 
of two histograms. In histogram A, one plots the distribution of the quantity of 
interest for all the data of the sample; in histogram B one plots the distribution 
of the same quantity, but only for those satisfying the selection criteria, i.e., 
those data that pass the cuts. Intuition leads one to expect that the best 
estimate for the (unknown true) efficiency of the cut for each bin is just ki/rii , 
where fej is the number of entries in bin i of histogram B and rii is the number 
of entries in bin i of histogram A. 

But what uncertainty should be assigned to each efficiency? For simplicity of 
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notation, from here forward let us consider only a single bin, in which k events 
out of a total of n events pass the cuts. To determine the errors in a histogram, 
one merely applies the same rule independently to each bin i. Please note that 
in the following we consider only statistical errors and ignore all aspects of 
systematic errors. 

2 Common but incorrect error evaluations 

There are two frequently used, but incorrect, approaches to evaluate the errors 
of the efficiency calculation. They are briefly discussed below, before we turn 
to the derivation of the correct treatment of the uncertainties. 

2.1 Poissonian errors 

Assuming that k and n fluctuate independently, the Poisson distribution in the 
large sample limit tells us that the error Uk in k, the output sample, is y/k, and 
that the error <j n in n, the input sample, is \fn. Then, using standard error 
propagation, and given the usual estimator for the real (unknown) efficiency 

E(e)=£=-. (1) 
n 

one finds for the variance: 

V( £ -)=.f = ^(i + i) (2) 
This calculation is flawed for various reasons: 

• n is a fixed quantity and not subject to any fluctuation. It's usually a well 
defined and known input quantity. This reduces Eq. [2] to 

f 2 k 

" 7 " (3) 

• Furthermore n and k are not independent but highly correlated. Statistics 
tells us that k must be distributed according to a Binomial probability 
density function. 

• Another strong argument against this method is the behavior in limiting 
cases. In the case k = and n > 1 one finds e = and ag, = 0. The 
calculation is telling us that if we observe one event, and it fails the cut, 
we know with complete certainty (zero error) that the efficiency is ex- 
actly zero. This is a remarkable conclusion, which differs greatly from our 
intuition. 

Clearly equation[3](and even more so[2]) are in disagreement with our reasonable 
expectations and basic statistics laws. We conclude that the Poissonian error 
evaluation is incorrect and is not applicable in our case. 
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2.2 Binomial errors 

Next let us consider a simple Binomial error calculation. This calculation is 
based on the knowledge that the application of a cut (or cuts) can be considered 
a Binomial process, with probability of "success" e, the true efficiency. Given 
this efficiency and the sample size n, the number of events passing the cut is 
given by a Binomial distribution 

P(k;e,n)= Q £ fe (l- £ )«- fe (4) 

with mean k = en, and variance V(fc) = uf. = ne(l — e). Using again the usual 
estimator for e — k/n, where n in our case is a given input constant we can 
write down the variance for e using simple error propagation: 

V (s) = 4 = = (5) 

n n 3 

However, this equation also yields absurd results in limiting cases. 

1. In the case k = we obtain an unphysical zero error as is the case in the 
Poissonian error calculation (Eq. [3]) . 

2. In the other limit, k = n, the formula again yields zero error. 

In each case, this calculation claims perfect certainty for the measured efficiency. 
Again, this violates our reasonable expectation. 

In the next section, we now develop a calculation, based on the use of Bayes' 
Theorem, that calculates the statistical uncertainty in the efficiency in a manner 
that agrees with our intuition, and that exhibits reasonable behavior even in 
limiting cases. 



3 Correct treatment of errors 

We start out again with the Binomial probability. P(k;e,n) defined in Eq. [?] 
denotes the probability that k events will pass the cut, given the conditions that 
the true efficiency is e, that there are n events in the sample, and that our prior 
information tells us this is a Binomial process. 

In our problem, we do not know e; rather, we have our data, which is an 
observation of k events out of n passing the cut. What we need to determine is 
the probability density function P(e; fc, n), which gives the probability function 
of e for a given n and k. Once known, we can determine easily the mean, 
variance, most probable value, and confidence intervals, so that we can make 
comparisons with some statistical meaning. 
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3.1 Derivation of the probability density function 



In order to calculate we use the Bayesian theorem and make the following ansatz: 

P( e ;fc,n)= P(fc;e '^ P(e;n) (6) 

where C is a constant to be determined by normalization, and P(e; n) is the 
probability we assign for the true efficiency before we consider the data. Given 
only n and the fact that we are dealing with a Binomial process says simply 
that e must be in the inclusive range < e < 1; we would have no reason to 
favor one value of the efficiency over another. Therefore it is reasonable to take 

P( ■ ) - { 1 if ° - £ ~ 1 (7) 
^ ' ' y otherwise 

independent of n. Probability theory allows us to include in our calculation the 
knowledge that the efficiency must be between zero and one; this knowledge 
is built into the predata probability distribution describing our knowledge of e, 
which assigns zero probability to those values of e which we know, with certainty, 
to be impossible. 

To determine the normalization we must solve 

P(e;k,n)de=^^f\ J e k (l - e) n ~ k dE = 1 (8) 

for C. For the calculation of the integral it is useful to recall the definition of 
the Beta function: 

B( a + M + l) = /'^(l-^ = r '° + '»^ 2 + / ) . (9) 
Jo T{a + (j + 2) 

Note also the trivial relation for integer values T(n + 1) = nl. Thus we directly 
obtain 

n' 1 

C = - - = —— (10) 
(n + 1)! n+l v ' 

and the final efficiency probability density function thus reads: 

P(e;k,n) = ^ + 1)^^(1-^ (11) 

- {n+1) - e k (l~e) n - k (12) 



kl (n-k)\ 

Figure Q] shows P(e; k, n) for n = 10 and k = 0, 1, . . . , 10. Note that in all cases, 
we assign zero probability that e is below zero or above one. Note also that we 
assign zero probability to e = unless k = 0; this is necessary, of course, since 
if we observe even a single event which passes, we know the efficiency cannot be 
zero. Similarly, we assign zero probability to e — 1 unless k = n, since if even a 
single event fails our cut, we know that the efficiency is not one. 
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Figure 1: The probability density function P(e; k, n) for n = 10 and k = 0, 1, . . . , 10. 

3.2 Features of the probability density function 

Given the analytic form we now can calculate the moments of the distribution. 
For the mean we obtain, again using Eq. [5] to solve the integral: 



e = 



/ e P(e; k, n) de 
Jq 

(n+1)! ^ 



kl(n-k)\J 
k + 1 



e fc+1 (l- e) n - k de 



n + 2 



(13) 
(14) 
(15) 



The most probable value (the mode) mode(e) can be easily calculated by solving 
dP/de = 0. We get: 



mode(e) 



(16) 



This is a remarkable result. The common estimator of the real efficiency e — 
k/n actually is the mode and not the mean, i.e., the expectation value of the 
distribution. The mean and the mode only become identical for n = 2k, and of 
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Figure 2: The probability density function P(e;k,n) for n = 100 and k 



course in the limit of large n. As depicted in Fig. Q] the density functions are 
skewed except for k = n/2 = 5. 

What to use if only one measurement was made: mean or mode? Statistics 
teaches us that an estimator cannot be described as 'right' or 'wrong' but only 
as 'good' or 'bad'. A good estimator has to be consistent, unbiased, and efficient. 
Clearly e — k/n has these features, while (k + l)/(n + 2) is biased for small n. 
In practice, however, this difference can be neglected since in most cases n is 
reasonably large. This is depicted in Fig. [2] for the case n = 100. 

Let's now turn to the original purpose of this exercise: the error evaluation. 
The calculation of the variance V(e) from P(e; k,n) yields: 



V{e) 



\ e 2 P(e;k,n) de -e 2 
Jo 

(fc+l)(fc + 2) (fc + 1) 2 



(17) 
(18) 

(19) 



(n + 2)(n + 3) (n + 2) 2 ' 
As expected V(e) now behaves correctly in the two extreme cases; for k = 
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Figure 3: Efficiency probability density function P(e; 8, 10). The solid vertical line 
depicts the mean value, the dashed line the most probable value. The gray shaded 
region corresponds to plus/minus one standard deviation (Eq. [19} around the mean. 



and k = n one finds 

n 4- 1 

V(e)\ k= o, n = - — >0. (20) 

(n + 2y (n + 3) 

For large n in this case the variance becomes lim V(e) = 1/n 2 . 

n — ^oo 

Interestingly the mean and variance for n = is non-zero. One finds e = 1/2 
and V{e)\ n= o = 1/12, which is simply the mean and variance of a uniform 
distribution in the interval from to 1. This is a nice confirmation of the 
validity of our calculations. The case n = essentially means that no prior 
information on the efficiency is available and all we can say beforehand is that 
the efficiency has to be between and 1. Since no efficiency is more likely than 
any other, the only assumption one can make is that of a uniform probability 
density. 

Figure [3] shows the probability density function for the case n = 10 and k = 8. 
The vertical dashed and solid lines depict the mode and the mean, respectively. 
In this case the region e ± a where a e = y/V{e) from Eq. [TSl corresponds to a 
confidence level (CL) of 67.3%. 
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3.3 Measure for the statistical error 

There are many measures of the uncertainty that can be extracted from the 
distribution P(e;k,n): upper and lower limits at various confidence levels; the 
variance, or its square root, the standard deviation; the mean absolute devia- 
tion; or confidence intervals of various sorts. Whatever choice, the situation is 
complicated by the fact that the function is (i) not symmetric and (ii) that the 
integral over P(e; k,n) is not analytic for limits other than and 1. 

There is no obvious recipe for cases where n is small and/or the efficiency found 
is close to or 100%. A reasonable measure is certainly the standard deviation 
a e . The advantage of this approach is clearly that a £ = y/Vje) can be easily 
calculated from Eq. [IT)] In addition it turns out that the confidence levels for 
'common' data sample sizes n and efficiencies are close to the "ler" probability 
content of the Gaussian distribution, i.e., 68.3%. This is of course no accident 
but a consequence of the Central Limit Theorem. Note that in the limiting 
cases, that is, small k, small n, or k close or equal to n, the confidence level 
deviates slightly from this value. 
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